REMARKS 

Claims 1-16, 18-25 and 28 are pending in the case. Claims 1-3, 13, 14 and 25 
have been amended and new claim 28 has been added. Claims 17, 26 and 27 have been 
canceled. No new matter has been added to the application. 

Claims 1-16 and 18- 24 have been rejected under 35 U.S.C. 103(a) as being 
unpatentable over Fukunaga, U.S. Patent No. 6,346,940. The Examiner stated essentially 
that Fukunaga teaches or suggests all the limitations of clauns 1 to 24. Applicants 
respectfully traverse the rejection. 

The present invention is directed to a method and system for augmented reality 
guided instrument positioning. A real view of an environment is displayed. A preferred 
path for positioning an instrument is determined. The preferred path is marked with a 
graphics guide. The real view is augmented with a rendering of the graphics guide such 
that at least one portion of the graphics guide is transparent with respect to other portions 
of the graphics guide to provide a substantially unobstructed view through the at least one 
portion of the graphics guide to at least a portion of the instrument. The instrument is 
aligned to the graphics guide so that the instrument appears in a same location as the 
graphics guide in the augmented view. When properly aligned, the instrument is visible 
through the at least one transparent portion of the graphics guide. The instrument is then 
inserted along the length of the graphics guide. The real instrument and virtual graphics 
guide appear in the same view for precise spatial alignment. 

Fukunaga teaches that a guiding marker preparation unit forms guiding markers 
(see col. 7, lines 65 to col. 8 line 3). Fukunaga does not teach or suggest "augmenting the 
real view with a rendering of the graphics guide" as claimed in claims 1 and 13 and 25. 
Fukunaga teaches an image processing system installed adjoining an electronic 
endoscope system (see col. 5, lines 30-35). Image processing system views are shown on 
a display imit while views from the electronic endoscope are shown on a separate monitor 
(see Figure 3). Fukunaga does not teach or suggest an augmenting the real view with a 
rendering of the graphics guide, essentially as claimed in claims 1, 13 and 25. Fukunaga 
only prepares a "map" by inserting guiding markers into a virtual model. Fukunaga does 
not teach showing a real view that includes the instrument to be positioned (in Fukunaga, 
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the instrument is an endoscope and captures the real view, thus not being visible itself), 
and Fukunaga does not teach to overlay the real view with the virtual map for precise 
spatial alignment (which, of course, cannot be done as the endoscopic instrument is not 
visible in the real view). 

Claims 1,13 and 25 are believed to be allowable for additional reasons. Fukunaga 
teaches that a degree of transparency of the virtual endoscopic image Ai is preset, but 
may be adjusted by an operator (see col. 11, lines 44-49). Fukunaga does not teach or 
suggest "at least one portion of the graphics guide is transparent with respect to other 
portions of the graphics guide to provide a substantially unobstructed view through the at 
least one portion of the graphics guide to at least a portion of the instrument" as claimed 
in claims 1,13 and 25. As the Examiner has suggested with respect to claim 3, Fukunaga 
fails to teach or suggest a transparent guide marker. Fukunaga treats the guiding markers 
separately from the endoscopic image Ai. Fukunaga teaches that guiding markers are 
placed on the virtual endoscopic image Ai (see for example, col. 1 8, line 38 to col. 19, 
line 60). The transparency of the endoscopic image Ai is set before the guiding markers 
are placed (see col. 19, Unes 30-35). Thus, Fukunaga does not teach or suggest 
transparent guiding markers. Indeed, Fukunaga fails to teach or suggest that anything 
other than certain virtual endoscopic images (e.g., images Hi as shown in Figure 15a) 
may be semitransparent. 

In Fukunaga, the guide markers support the user to find the right way along a 
branching system of "chaimels" (e.g., bronchial tree). The guide markers are not 
intended to mark exact spatial endoscope locations inside the channels. In contrast, the 
present invention uses the virtual graphics guide in the context of an augmented view to 
precisely mark where a real instrument has to be placed. The user has to perceive the 
exact spatial relationship between the virtual graphics guide and the real instrument, both 
visible in the augmented view, and bring the instrument (e.g., needle) into correct 
alignment with the graphics guide. 

Furthermore, Fukunaga fails to teach or disclose "aligning the instrument to the 
graphics guide so that the instrument appears in a same location as the graphics guide in 
the augmented view, and when properly aligned, the instrument is visible through the at 
least one transparent portion of the graphics guide; and inserting the instrument along the 
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length of the graphics guide". Fukunaga describes a variety of visualization options. To 
see behind the surface of the virtual model or beyond a graphics guide, Fukunaga renders 
the guide semi-transparent or as a wirefiame. 

In contrast, the simple visualization options of Fukunaga are not sufficient to 
correctly perceive the alignment of a real instrument and a graphics guide in an 
augmented view. The core problem is that of incorrect occlusion. The virtual guide 
always occludes the real instrument; correctly when the real instrument is indeed behind 
the guide, but also incorrectly when the real instrument is actually in front of the virtual 
guide. Semitransparent rendering of the guide avoids complete occlusion of the real 
instrument; however, even the partial occlusion of the real instrument by the virtual guide 
is wrong when the real instrument is in front of the guide, and still interferes with the 
correct spatial perception. 

The present invention introduces the concept of virtual guides. Along the length 
of the guide, the transparency of the guide is modulated. By this it is meant that there are 
sections where the guide "disappears" to provide an essentially unobstructed view onto 
the instrument during the alignment procedure, and other sections where the guide is 
clearly visible. This modulation introduces a completely new quality to the visualization 
of the virtual guide, and to the perception of the alignment of the real instrument and 
virtual guide in the augmented view. 

Fukunaga fails to teach or suggest an augmented view as an overlay of a real and 
a virtual view, a transparent guide marker, modulation of a guide marker's transparency, 
and alignment of a real instrument and a virtual graphics guide. In light of Fukunaga's 
deficiencies, one of ordinary skill in the art would not be lead to augmenting the real 
view with a rendering of the graphics guide having at least one portion of the graphics 
guide being transparent with which the real instrument is aligned, essentially as claimed 
in claims 1,13 and 25. Therefore, Fukunaga fails to teach or suggest all the limitations of 
claims 1, 13 and 25. 

Claims 2-12 and 28 depend from claim 1. Claims 14-16 and 18- 24 depend from 
claim 13. The dependent claims are believed to be allowable for at least the reasons 
given for claims 1 and 13. Applicants respectfully request that the rejection of claims 1- 
16 and 18-24 under 35 U.S.C. 103(a) be withdrawn. 
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For the forgoing reasons, the present application, including claims 1-16, 18-25 
and 28, is believed to be in condition for allowance. The Examiner's early and favorable 
action is respectfully urged. 
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